Agenda

Note: See chapter 3 and 4 from the text

Observability (activity review)

We installed the kubernetes dashboard. From their github:

Kubernetes Dashboard is a general purpose, web-based UI for Kubernetes clusters. It allows users to manage applications running in the cluster and troubleshoot them, as well as manage the cluster itself.

Kubernetes dashboard

kubernetes dashboard
kubernetes dashboard

Deployment diagram

deployment diagram
deployment diagram

Telemetry Lab

Create a codespace from the github template and run:

pip install -r requirements.txt
ansible-playbook playbook.yml
kubectl port-forward svc/my-otel-demo-frontendproxy 8080:8080

Follow instructions in chapter 4 of the text and make notes in README.md. Submit the README.md file with the uploader at the end of the presentation.

Learning outcomes

A Brief History of Telemetry

Southern Railway, 1928
Southern Railway, 1928

“Telemetry” sent over telegraph lines

early but important distributed systems!

computer telemetry on network

Supervisory Control And Data Acquisition (SCADA)

MTA Capital Construction Mega Projects, CC BY 2.0 https://creativecommons.org/licenses/by/2.0, via Wikimedia Commons
MTA Capital Construction Mega Projects, CC BY 2.0 https://creativecommons.org/licenses/by/2.0, via Wikimedia Commons

Uses

Critical Infrastructure

Simple Network Management Protocol (SNMP)

An.bellizzi, CC BY-SA 4.0 https://creativecommons.org/licenses/by-sa/4.0, via Wikimedia Commons
An.bellizzi, CC BY-SA 4.0 https://creativecommons.org/licenses/by-sa/4.0, via Wikimedia Commons

SNMP consists of three key components

Uses

  • SNMP used by IT to monitor and update networked devices
  • SCADA used by operations to control processes

Open Telemetry

Chapter 4 and the open telemetry demo
Chapter 4 and the open telemetry demo

Open Telemetry by contrast

Time Series Database (TSDB) Storage

Store enough to meet stakeholder needs

Thanos provides a global query view, high availability, data backup with historical, cheap data access as its core features in a single binary.

Storage Configuration

Otel collectors

      - name: OTEL_EXPORTER_OTLP_TRACES_ENDPOINT
        value: http://$(OTEL_COLLECTOR_NAME):4318/v1/traces

Telemetry Example (Trace)

explore/jaeger/adService …
explore/jaeger/adService …

Manual testing for observability

Observability Coverage

Telemetry

Without telemetry, your system is just a big black box filled with mystery.